Counterfeit Medicine Detection by Visual Inspection of Package Design using Multimodal LLMs with Text and Image Prompt Engineering

Yona Zakaria, Eiki Ishidera, Rui Ishiyama, Tomokazu Matsui, Hiroiko Suwa, Yuki Matsuda, Keiichi Yasumoto: “Counterfeit Medicine Detection by Visual Inspection of Package Design using Multimodal LLMs with Text and Image Prompt Engineering,” The 17th International Conference on Intelligent Medicine and Image Processing (IMIP '25), Vol.13658, pp.92-99, 2025.

Abstract

This study explores the potential use of multimodal large language models (LLMs) in detecting counterfeit drugs through visual inspection of medicine packaging designs. Specifically, we investigate how ChatGPT-4o can provide clear explanations of design differences between genuine and counterfeit packaging. We combine structured textual prompts with three distinct image configurations: (1) query-only images, (2) query-plus-reference images, and (3) query-plus-reference-plus-difference. This setup allows for context-aware comparative analysis, helping the model to effectively identify and explain packaging design inconsistencies—key indicators of counterfeit or substandard medicines. Experimental results show that ChatGPT-4o achieves a binary classification accuracy of up to 74.6% in distinguishing authentic from counterfeit medicine packaging. Furthermore, user evaluations reveal that ChatGPT-4o delivers high levels of clarity, ease of understanding, reliability in identifying discrepancies, detail, and overall quality of analysis. These findings underscore the notable potential of ChatGPT-4o to enhance explainability and usability in counterfeit detection workflows, particularly by enabling accurate, actionable insights without requiring training on counterfeit-specific datasets, which are often challenging to collect.

Links

DOI: https://doi.org/10.1117/12.3071640

PDF: https://cocolab.jp/publication/files/202504_IMIP_Yona.pdf

BibTeX

code:references.bib

@inproceedings{bib:yona_fakeMedicineDetection_IMIP2025,

author={Zakaria, Yona and Ishidera, Eiki and Ishiyama, Rui and Matsui, Tomokazu and Suwa, Hirohiko and Matsuda, Yuki and Yasumoto, Keiichi},

title={Counterfeit Medicine Detection by Visual Inspection of Package Design using Multimodal LLMs with Text and Image Prompt Engineering},

booktitle={The 17th International Conference on Intelligent Medicine and Image Processing (IMIP '25)},

volume={13658},

pages={92--99},

year={2025},

doi={10.1117/12.3071640},

url={https://doi.org/10.1117/12.3071640}

}

https://scrapbox.io/files/68ca6fa9706354bf85bee373.png